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Abstract 

Identity of a vehicle is done through the vehicle license plate by traffic police in general. Au¬ 
tomatic vehicle license plate recognition has several applications in intelligent traffic management 
systems. The security situation across the globe and particularly in India demands a need to equip 
the traffic police with a system that enables them to get instant details of a vehicle. The system should 
be easy to use, should be mobile, and work 24 x 7. In this paper, we describe a mobile phone based, 
client-server architected, license plate recognition system. While we use the state of the art image 
processing and pattern recognition algorithms tuned for Indian conditions to automatically recognize 
non-uniform license plates, the main contribution is in creating an end to end usable solution. The 
client application runs on a mobile device and a server application, with access to vehicle informa¬ 
tion database, is hosted centrally. The solution enables capture of license plate image captured by the 
phone camera and passes to the server; on the server the license plate number is recognized; the data 
associated with the number plate is then sent back to the mobile device, instantaneously. We describe 
the end to end system architecture in detail. A working prototype of the proposed system has been 
implemented in the lab environment. 


1 Introduction 

License Plate Recognition (LPR) systems are usually designed to read vehicles license plate and auto¬ 
matically recognize license plate number in ASCII, of vehicles passing through a certain point. Systems 
are readily available for mass surveillance that utilizes optical character recognition (OCR) and hard¬ 
ware capable of reading license plates of moving vehicles 01 2, 3 ] but still continue to hold the interest 
of researchers 0 5, 6,17, 8 , 9, T0llTTllT2l mainly because of several challenges that exist. Low quality 
images due to severe illumination conditions, vehicle motion, viewpoint and distance changes, com¬ 
plex background are some of challenges. LPR is preceded by localization of license plate. ifTTI use a 
feature-based license plate localization algorithm that copes with multi-object problem in different im¬ 
age capturing conditions which they claim is robust against illumination, shadow, scale, rotation, and 
weather condition. Vahid et. al. [5] address some of these problems by using intensity variance and edge 
density image enhancement methods. More recently Luis et al 0| have suggested the use of artificial 
neural networks for license plate detection and Cristian et al (6) suggest a mechanism to fuse decisions 
for improving the license plate recognition. Some systems make use of infrared cameras to increase the 
efficiency of the system. LPR systems can be used for vehicle identification JT3), enforcement, collect 
electronic tolls lfl4ll . traffic monitoring and travel management lfl5l . In all these systems the camera is 
fixed and is therefore only able to scan the vehicle passing through a particular point. On the other hand, 
mobile LPR system has become a necessity for law enforcement especially with increasing volumes of 
vehicles being added to Indian roads every year. 

Traffic police not only make sure that the vehicles on the road follow traffic rules but also make 
sure that the vehicles have necessary authorizations to be on the road. In any large city the traffic police 
inspectors can police a fraction of the city and when ascertaining details of the vehicle have to rely solely 
on the details provided by the vehicle driver. In this paper we address the problem of equipping the traffic 


1 


Mobile License Plate Recognition 


Lajish, Kopparapu 



Vehicle 


Figure 1: The high-level client-server architecture of the solution. 


police with a mobile tool that can be used to get on the spot exact details of a vehicle on the road 24 x 7. 
We propose a mobile phone based vehicle license plate recognition system to assist the traffic police. The 
mobile phone based LPR has several applications. They could be used to identify (a) vehicles involved in 
road accidents, (b) trace stolen vehicles, (c) trace VIP escorting vehicles, (d) monitor and issue memo for 
violation of traffic rules. The paper is organized as follows: in Section[2]we describe the overall system 
architecture of the solution. Experimental setup and results are discussion of Section [4] and conclude in 
Section [5l 


2 The Solution Architecture 


The proposed solution has two parts (a) mobile client which enables capture of the license plate of a 
vehicle and send the image to a remote server and (b) a remote server which has a LPR software and 
access to an external database which has vehicle number and vehicle information association details. 
Figure [I] shows the high-level client-server solution architecture. 

The major client side components (Figure |2(a)| ) are (a) a server interface module and (b) a media 
interface (capture image) module. The client module communicates with the server using the HTTP. In 
a typical working scenario, the client application requests the user to capture the image of the vehicle 
license plate and enables the camera capture mode on the mobile phone. Once the image is captured, 
the image is pushed to the server for recognition. On receiving information from the server, the client 
software displays the obtained information about the vehicle as a text message to the user. Figure |2(a)| 
shows the client side solution architectures in detail. 

Note that the client application can be downloaded over the air (current implementation is on BREW 
but can be enabled for multiple mobile devices and multiple OS). 

The major components on the server (Figure [2(b)] ) include (a) image processing and pattern recog¬ 
nition modules for recognizing license plate and (b) a database interface. The server can interact with 
external databases (for example, in case of stolen vehicles IT6l ) and find the association between vehicle 
number and its status against a stolen vehicle. The ownership details (name, contact address, number), 
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Figure 2: Solution Architecture Details 


vehicle details (make, model, engine number etc.), tax details and previous complaints registered against 
the vehicle can also be obtained if a database is available. The server works on the image sent by the 
client while maintaining a link between the image uploaded and the mobile device from which the image 
was uploaded through a unique session number. A module to recognize vehicle registration number from 
the image forms the heart of this solution and is described in greater detail in Section [3] 


3 License Plate Recognition 

The LPR engine resides on the server and is a hub of several image processing and pattern recognition 
modules. The important functional units include image pre-processing, license plate localization, char¬ 
acter segmentation and character recognition ifTTl [T8l . Figure [3] shows the block diagram of the LPR 
engine. After initial pre-processing of the LP image, the next task is that of locating the exact position of 
the LP in the image (LP localization). Once the LP is located the actual LPR happens. 


Captured 
Number ~~ 
PJate Image 



Figure 3: Block diagram of LPR engine 

Actual sample images of vehicle license plates captured by mobile phone cameras are shown in 
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(e) (f) (g) (h) 

Figure 4: Images of the license plates taken using mobile phone cameras 



(a) Input (b) Output 


Figure 5: Sample vertical edge output 


Figure [4] It can be observed that the input images have a great deal of variability in terms of variability 
in lighting conditions, shadows on the plate region, plates from front or rear part of the vehicle with 
messages or icons, skewed images, background and foreground of the number plate and one line and two 
line writing, the recognition system needs to work under all these conditions. 

License Plate Pre-processing: The color image captured from the camera is first converted into a gray 
scale image and is then binarized using Otsus global thresholding technique fl9l . Note that we could use 
an adaptive threshold technique to get a binary image but our analysis with a range of LP images showed 
that Otsus thresholding technique works well. We also use, skew correction up to about ±5 degrees and 
slant normalization before license plate localization is carried out. 

License Plate Localization: Determining the location of the LP within the captured image automatically 
is a critical task for successful recognition. The aim is to obtain a region of interest (usually a rectangular 
window) in the image (say /(x,y) of size M x N) that includes the license plate of the vehicle. For this 
purpose, the edge map E(x,y) of I{x,y) is initially computed by applying the Sobel edge operator in 
the vertical direction since presence of characters contribute more edges in vertical direction (see Figure 
[5]). Note that the edge values are real numbers. We eliminate all but the top 3% of the edge intensity 
pixels and obtain a reduced binary edge map E(x,y). We compute variance (crj(x)) of the each row 
of the edge image (Z?(jc,;y)) and calculate the maximum variance (<7^ x ). We select all the rows (x) 
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with ot{x)> 0.5 We mark and collect contiguous rows as regions (say this isolates 

the license plate in the horizontal direction. To perform vertical cropping of the selected regions, we 
examine the edge strength (by observing the vertical profile of the horizontally segmented region) in 
vertical direction (E{x,y) in the y direction). The columns with 50% of the maximum edge strength are 
retained and the co-ordinates of the license plate region are identified. We end up with one or more 
candidate license plate regions. 

Character Segmentation: After identifying the regions of interest the next step is to segment the char¬ 
acters from the LP as a spatial sequence of alpha-numeric number (which gives the exact number of 
the vehicle). Segmentation of the characters written in different formats (single line, double line, or in 
different orientation) is a challenge. The character segmentation (isolation) process involves use of con¬ 
nected component analysis to segment the characters EOl . Post processing on the identified connected 
components is performed to eliminate non-character like logos. Post processing involves use of spatial 
position of the connected components (all the characters have more or less the same x coordinates but 
varying y coordinate), the aspect ratio of the bound box on the connected component (all the characters 
are more or less of the same size) and also use of high level information like the maximum number of 
characters expected in the license plate, etc. The isolated characters are then size normalized using affine 
transformation and bilinear interpolation techniques [2JL]. 

Character Recognition: The segmented characters are recognized using template matching technique. 
Here the segmented character is compared with all the reference character set and the similarity is com¬ 
puted (correlation coefficient). The reference character with the highest correlation is chosen as the 
identified character. Note that the template database consists of all English capital letters and Arabic 
numerals. Algorithm [I] describes the proposed solution. 


Algorithm 1 Steps in LPR 

1: Convert Colour (7 c (x,y)) image to Grayscale (I(x,y)) 

2: Extract vertical edge map using Sobel edge detection algorithm (£ v (x,y)) 

3: Compute variance (aj(x)) of the each row of the edge image and compute the maximum variance 

(a 2 ) 

4: Select all the rows (x) with cj|(x) > 0.5 x o 2 ax 

5: Mark and collect contiguous rows as regions (say , <^----) 

6: Binarise all the selected region (/^ ,/^ 2 , • • •) using Otsu’s algorithm 
7: Perform vertical cropping of the identified region based on vertical profile 

8: Apply morphological dilation for bridging the discontinuities within the character segments in the 
plate region. 

9: Perform character segmentation based on connected components 

10: Perform rule (area /aspect ratio / existence ratio) based elimination of the non character components. 


4 Experimental Results 

There is no standard test bed, to the best of our knowledge, to test the performance of a LPR system; 
hence we collected our own data. Additionally, we needed to work on mobile camera resolution images. 
We collected images as follows: (a) the images are colour images of actual Indian license plates taken 
under various conditions, (b) all images are taken in 640 x 480 resolution or lower with different mobile 
phones at our disposal. The database contained a total of 871 images of 230 different vehicles. The 
results of image processing algorithms used in the recognition including plate localization, binarization, 
character segmentation and size normalization are given below. 
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Figure 6: Some Experimental Results 


Sample experimental outputs shown in Figure [6] Results show that the license plate recognition 
engine is capable of handling images which are skewed and also able to handle when license plate 
characters come in more than one row. Experiments were performed to measure the accuracy of LP 
region of interest extraction and the LP recognition. The complete results will be presented in the final 
version of the paper. 


5 Conclusions 

In this paper, we presented the solution architecture and a sequel of algorithms for the recognition of LP 
of a vehicle in Indian context using mobile phone camera images. This system has been tested over a 
large number of real images with very encouraging results. The proposed system is fast and yields robust 
recognition results Almost all existing vehicle license plate recognition systems use one or more fixed 
cameras which put a restriction on mobility. In contrast, the system proposed uses mobile phone camera 
to capture license plate images, meaning we have a vehicle plate recognition system which is mobile (we 
can take it to any location). Mobility facilitates handling vehicles parked anywhere or when vehicles are 
stopped for inspection by the traffic police. We believe, mobile phone based systems are currently not 
available anywhere in the world and especially for Indian license plates. Comparison with any existing 
LPR system is difficult because of differences in the working environment of the proposed system with 
a existing LPR system. As a future enhancement, location information can be captured along with the 
LP images and can be used in vehicle tracking application. The average speed and travel time between 
two points can be calculated and presented in order to monitor traffic flow and load in a specific area of 
a city. 
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